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Abstract 


Recursive  algorithms  for  linear  least  squares 
estimation  problems  have  been  based  mainly  on 
state-space  models.  Recently,  some  new.  recursive 
solutions  were  obtained  for  processes  classified 
In  terms  of  their  "Index  of  nonstationarity"  or 
equivalently — the  displacement  rank  of  their 
covariance  functions.  While  this  definition 
provides  a natural  explanation  of  the  properties 
of  constant-coefficient  state-space  model's.  It  Is 
not  satisfactory  for  time-variant  models.  How- 
ever a modified  definition  of  Che  displacement 
rank  makes  It  possible  to  Imbed  time-varying 
state-space  models  in  the  more  general  Input-out- 
put framework.  In  so  doing,  we  are  able  to  show 
the  mutual  relationships  of  the  Kalman  filter 
Rlccati  equation,  the  time-varying  Chandrasekhar 
equations  and  Che  Kreln-Levinson  equations. 

• . . 1 i-s. 

I.  Introduction 

In  [l)-[5]  we  have  developed  recursive  esti- 
mation algorithms  using  input-output  models  (e.g. , 
covariance  functions)  instead  of  state-space 
models.  A central  Idea  In  our  approach  was  that 
of  the  displacement  rank  a (an  "Index  of  non- 
statlonarity")  of  the  covariance  functions  of  the 
signal  and  observation  processes.  Using  this 
notion  certain  Sobolev-  and  Kreln-Levinson-type 
differencial  equations  were  developed  for  the 
optimal  smoother  and  optimal  filter,  leading  to 
computational  algorithms  whose  complexity  depends 
on  the  displacement  rank  a. 

By  imposing  constant-parameter  state-space 
structure  on  Che  covariance  functions,  we  Chen 
showed  In  [1]  how  the  Kreln-Levinson  equations 
led  to  the  Chandrasekhar  equations  for  the  compu- 
tation of  Che  Kalman  gain.  The  successful  im- 
bedding of  Che  state-space  case  in  the  Input-out- 
put  framework  was  due  to  the  fact  that  the 
covariance  function  of  constant  parameter  state- 
space  models  has  a (relatively)  small  displace- 
ment rank.  However,  processes  associated  with 
time  varying  models  will  not  have  small  a and 
may,  in  fact,  have  Infinite  displacement  ranks. 
This  fact  prevented  us  from  treating  time  varying 
models  In  our  input-output  framework. 


In  Section  II  we  shall  show  that  this  dlffl- 
culty  can  be  circumvented  by  Introducing  a more 
. general  definition  chat  (1)  leads  to  small  values 
of  a both  for  time-varying  and  time-invariant 
state-space  models,  (11)  coincides  with  the 
previous  definition  of  a In  the  time-invariant 
case.  In  Sec.  Ill,  we  note  briefly  how  the  general 
results  of  [ll-(2]  will  be  modified  with  the  new 
definition  of  displacement  rank — the  changes  are 
minor.  Then  in  Sec.  IV  we  shall  show  bow  the 
Imposition  of  state-space  structure  leads  to  the 
time-variant  Chandrasekhar  equations  of  [6].  A 
direct  derivation  of  these  equations  was  first 
given  In  [7]-[8]  (see  also  I9)-I10)).  Unfortu- 
nately, as  noted  In  [8]-[9]  the  time-variant 
version  is  just  a set  of  two-point  boundary  value 
equations,  which  Is  not  especially  easy  to  solve. 

In  fact;  a standard  approach  to  two-point  equations 
is  via  the  Rlccati  equation,  and  we  shall  show 
] that  this  can  be  done  here  as  well.  Of  course, 

I the  Rlccati  equation  could  also  have  been  directly 
: obtained  from  the  state-space  models,  (as  in  the 
usual  Kalman  filter).  The  contribution  here  is 
that  we  show  how  to  deduce  the  Rlccati  equation 
, from  a more  general  set  of  equations  applicable 
when  no  state-space  models  are  available. 

Most  proofs  are  omitted  in  this  short  paper, 

: but  may  be  found  In  [11];  there  we  also  indicate 
' the  analogous  results  for  discrete-time  estimation. 

, II.  The  displacement  rank  of  covariance  functions 

The  displacement  rank  of  a kernel  K(’,-)  is 
defined  in  [1]  as  the  smallest  integer  a such 
that  we  can  write  ^ 

(|^  + |j)K(t,8)  - K(t,T)  K(t,8)  +D(t)AD'(s)  (1) 

where  K,  D,  and  A are  matrix  functions  with 
dimensions  p x p,  p x a and  a x a respectively. 
The  functions  D and  A need  not  be  unique, 
though  it  will  often  be  simplest  to  assume  chat 
A is  diagonal. 

The  reasons  for  choosing  this  definition  of  the 
displacement  rank  and  its  application  In  solving 
Fredholm  and  Wlener-Hopf  type  Integral  equation 
arc  discussed  in  detail  in  [1],  [2]  and  will  not 
be  repeated  here. 

Instead  we  shall  focus  on  Che  special  case  of 


processes  generated  by  lumped  state-space  models 

x(t)  - F(t)x(t)  + G(t)u(t)  , x(t)  •»  X.J. 

y(t)  - H(c)x(c)  + v(t)  , t T 

where  x(*)  is  nxl,  u(-)  Is  mxl,  and  y(') 
scalar, 

E u(t)u'(s)  " Q(t)6(t-s),  E v(t)v'(s)  ■ I 6(t-8) 
E u(t)v'(s)  = 0 H E u(t)x.J  , 

and' 
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where  ♦ la  the  atate  tranaltlon  matrix  oi 
It  la  obvious  from  (5)  that  the  new  dlaplacem«t 
rank  la  upper  bounded  by  n.  Note  also  that  even 
In  problems  where  there  la  no  dependence  on  T 
(aay  a K(t,s)  defined  for  0 < t.a  < T),  the  n* 
definition  can  atlll  be  uaed  by  Introducing  T 
artificially, , say  by  .K{t,o)_  “_K(t-T,  s-t)]^  ^ 
so  that  I i 

ij:  K(t-T,  ^8-r)  , 0 “ ^It 

whlch'coincides” wlth“the"origlnal  definition. 

ill.  The  modified  Sobolev-  an£  Levlnson-Kreln- 

Ti  ^ii^-'.Ttype'  equations  „j,  iir;r 

Let  us  now  consider  the  problem  of  estimating 
a stochastic  process  x(*)  (n-dlmenslonal)  from 


a given  nxn  matrix. 


When  the  model  parameters  IFl*),  Gf*},  nl*J, 
Q(*)}  are  constant  then  It  was  shown  In  £lj  that 


In  which  case  o - 1 . For  other  Initial  conai- 
tlons,  a will  generally  be  greater  than  1,  but 
la  any  case  we  shall  always  have  O £ n.  The 
significance  of  a Is  that  n(l-*a)  is  the  • 
nuiber  of  equations  in  the  Chandrasekhar  equa- 
tions for  finding  the  least-squares  estlMtes  of 
x(*)  , which  is  to  be  compared  to  the  n^/2 
equations  needed  In  the  usual  Riccatl-equatlon- 
based  Kalman  filter  solution.  We  refer  to  [IJ 
for  more  details. 

Our  interest  here  Is  In  the  fact  that  when 
F(t),  H(t),  Q(t)  are  time-varying,  applying  the 

operator  + |j)  to  the  covariance  function 

will  lead  to  a rather  Involved  expression.  We 
can  no  longer  claim  that  the  displacement  rank 
of  K(t,s;T)  will  be  bounded  by  n and  It  may 
even  be  Infinite.  This  raises  the  question  of 
using  a different  definition  of  the  displacement 
rank,  one  that  will  still  yield  an  upper  bound 
of  n for  the  displace  rank. 

Let  us  re-deflne  the  displacement  rank  of 
K{«,*)  as  the  smallest  integer  a such  that 

- 1^  K(t,s;T)  - K(t,T)K(T,s)  +D(t)A  D'(s)  (3) 

where  K , D and  A have  dimensions  pxp,  pan 
and  0x0,  respectively.  To  see  why  this  particu- 
lar definition  was  chosen  note  that  In  the  con- 
stant parameter  case  the  covariance  K depends 
only  on  the  difference  (t-T)  and  (s-t),  which 
can  be  symbolically  written  as  K(t,s;T)  ■ 


where  D , A are  as  defined  earlier,  and 
Is  defined  by  the  equation  above.  Using  out  new 
definition  for  the  displacement  rank  we  shall 
replace  (9)  by 

-|-K  (t,8)-K  (t,T)K(T,8)-lD  (t)AD‘(s).  (10) 


be  shown  that  with  the  new  definitions, 
;aln  a very  similar  set  of  equations  for 
to  those  presented  In  (1).  in  fact, 
:hangcs  • that  have  to  be  made  are  replac- 

■I-)  by  (-  |r)  using  the  new 
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»Jt;8) +/*  K(8,o)n(t;o)do  - Dls)  , <12a) 

B (t;s)  +jr‘  K (s.o)B(t;o)da  • D (•)  (12*>) 

B^(t;t)  - D^(t)  - hj^y(t,0)D(O)dO  . (12c) 

Notice  that  the  dependence  of  h^(t,s)  on  T 
is  not  explicitly  shown. 

IV.  Imbedding  the  state-space  case  in  the  Input- 
output  framework. 

Let  us  now  Introduce  some  further  assumptions 
about  the  covariance  functions  K , , which 

will  impose  a state-space  type  structure  on  the 
processes  x(*)  and  y(*).  We  assume  that  there 
exist  functions  F(*)  (n  x n)  aiid  H(.)  (p  x n) 
such  that  K(t,s)  •>  H(t)K^(t,s),  and 

K^(t,8)  - F(t)Kj^y(t,8) . It  can  be  shown  [IJ 

that  under  these  assumptions  the  filtered  esti- 
mate Si(t)  obeys  the  usual  Kalman  filter  equation 

^S(t)  - F(t)a(t) +h_(t,t)(y(t) -H(t)a(t))  . 
oc  xy 

where  h^(t,t)  can  be  identified  as  the  Kalman 
gain.  TM  significance  of  this  fact  is  that,  at 
least  in  the  constant-parameter  case,  it  was 
shown  (1),  17]  that  the  Chandrasekhar-type  equa- 
tions could  be  used  to  compute  hj^(t,t),  instead 
of  having  to  compute  h^(t,s)  for  all 
T < 8 < t as  would  be  required  when  no  state- 
spTce  Structure  is  available. 

We  shall  now  show  what  are  the  corresponding 
equations  for  the  time-varying  case.  With  the 
assumptions  (13),  we  see  from  (12a, b)  that  we 
can  identify 

B(t;8)  •>  H(8)B  (t;8) 

xy 

and  therefore,  we  can  write  (11)  as 

‘ It  B;^(t;t)H'(t)  (U) 

With  a little  calculation  (see  [11]  and  (!]), 
it  can  also  be  shown  that  ) 

|jB^(t;t)  - (F(t) -h^y(t,t)H(t))B(t;t).  (15) 

Equations  (14),  (15)  are  the  time-varying 
version  of  the  Chandrasekhar-type  equations  that 
were  derived  using  a completely  different 
approach  in  [8],  (9].  Note  that  (14),  (15)  can 
not  be  solved  directly  since  the  time  arguments 
"do  not  fit”,  chat  Is,  because  of  the  opposite 
directions  of  evolution  of  these  equations,  at 
any  intermediate  point  the  values  of  Bxy(-;0 
needed  to  solve  for  h^y(',*)  , will  not  be 
available.  This  difficulty  is  circumvented  in 
the  time-invariant  toodel  case,  because  now  we 
can  reverse  the  direction  of  time  in  (14)  since 
in  this  case 

h “ ■ It  '‘xy^'*‘^  • 

In  the  time-variant  case,  the  Chandrasekliar 
equations  (first  obtained  in  [8])  have  to  be 


regarded  as  a general  set  of  two-point  boundary- 
value  equations,  with  all  the  attendant  difficult 
tics.  It  Is  wellknown  that  the  Rlccati  equation 
enables  us  to  replace  the  two-point  Hamiltonian 
equations  of  control  and  estimation  theory  by  an 
initial-value  equation,  and  this  can  be  done  here 
as  well. 

Introduction  of  the  Rlccati  Equation 

This  difficulty  can  be  resolved  by  considering 
instead  a different  quantity  P(t,T),  defined  by 

“ It  -B^y(t;t)AB*^  . 

• (t;t)H'(t)  , (16) 

or  rather,  in  its  integrated  form  h^(t,C)  ■ 
P(t,T)H'(t).  Diffe.rentatlon  of  (16)  with  respect 
to  t and  integration  with  respect  to  T gives 

|^P(t,T)  - F(t)F(t,T) +P(t,T)r’(t) -P(t,T)H'(t)  • 
• H(t)P(t,T) +Q(t)  , P(T,t)  - P(t) 

(17) 

Q(*)  being  Che  integration  constant.  This  is  an 
initial-value  equation,  solution  of  which  will 
yield  PCt,T)  and  then  hjjy(t,t)  via  (16)  and 
then  lc(t)  via  Che  Kalman  ^filter  equation. 

In  fact,  we  have  now  obtained  the  usual  Kalman 
filter  solution,  except  that  we  have  not  yet 
identified  Q(*).  In  Che  case  Chat  x(>)  and 
y(-)  are  related  by  the  state-space  nodal  of 
Sec.  II,  it  can  be  shown  that  9(*)  > 

G(*)  Q(*)  G'(*),  as  expected. 

V.  Concluding  Remarks 

We  presented  a new  definition  for  the  displace- 
ment rank  of  covariance  functions  that  makes  it 
possible  to  Imbed  both  time-invariant  and  time- 
varying  state-space  models  In  a more  general 
input-output  framework.  This  approach  provides 
Insight  into  the  relationship  between  various 
solutions  to  the  estimation  problem  and  clarifies 
the  role  of  Che  state-space  structure  in  simpli- 
fying the  estimation  algorithms.  For  processes 
with  measure  a of  "distance  from  stationaricy” 
recursive  estimation  algorithms  of  Che  Levinson- 
type  can  be  derived  [1],  (4).  If  additional 
(state  space  type)  structure  is  added  to  Che 
problem,  alternative  algorithms  become  available. 
In  the  time-varying  state-space  case  the  Kalman- 
filter  and  the  Rlccati  equation  are  naturally 
obtained,  while  in  the  constant-parameter  case 
the  more  efficient  Chandrasekhar  equations  may  be 
used.  Of  course  the  general  input-output 
recursions  have  to  be  used  when  state-space 
models  are  not  readily  available. 
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